Repository navigation
YOLO end-to-end example, CircularTensor hardening, CPU backend, ThreadFusion, multi-GPU routing - #2
Merged
morousg merged 3 commits intoJun 11, 2026
Conversation
…ssing examples/11_yolo_inference.py: JPEG -> ONE fused FKL kernel -> YOLOv8n. The preprocessing that ultralytics does in several numpy/torch steps is a single fused kernel: TensorRead (uchar3, any size) -> Resize(640x640, bilinear, PRESERVE_AR letterbox with the YOLO gray-114 padding) -> Div(255) -> TensorSplit (HWC -> planar CHW float, NCHW-ready). Standard decode+NMS postprocessing in numpy; inference via onnxruntime. Verified on bus.jpg (810x1080): 4 persons + 1 bus detected with correct boxes mapped back through the letterbox transform. Steady-state fused preprocess: ~0.55 ms (incl. D2H); kernel compiles once then disk-cached. examples/fetch_yolo_assets.sh downloads bus.jpg + a COCO-pretrained yolov8n.onnx; assets/ and .venv-yolo/ gitignored.
…ce tests REAL BUG found while hardening: the generated ct_update/ct_snapshot baked self.width/self.height as C literals, but the compile-cache signature only contains dtype/batch/order/layout/chain — NOT the size. Two CircularTensors with the same signature and different sizes would cache-hit the first .so and run with the WRONG hardcoded grid and snapshot byte count (out-of-bounds reads/writes; surfaced as 'invalid argument' in an unrelated test). Fix: PyCT now stores w/h at ct_create time and the kernel launch + snapshot use them at runtime, matching the values-vs-types rule everywhere else. CODEGEN_VERSION 6->7. New CircularTensor tests (21 checks total now): - t_race_audit: proves the divergent update is race-free by construction -- the temp slot written by seq1 is never among the slots read by seq2, for both orders and every rotation index. - t_temp_data_consistency: temp/data double-buffer coherence across 3 full rotation wraps, verified plane-by-plane via ct_snapshot_temp. - t_generate_source_no_compile: new generate_source() API (codegen without compiling) checked for all structural pieces. - t_size_not_in_signature_regression: the exact bug scenario. CI: the codegen-compile-check job now also compiles CircularTensor sources (packed/planar, newest/oldest, with preproc chains) via generate_source().
…n tests Closes out the pending-work checklist: - CPU backend (target='cpu'): generates a plain C++ TU running TransformDPP<ParArch::CPU> on MemType::Host pointers, compiled by clang++ (g++ rejects Stream_<ParArch::CPU>'s ctor spelling), numpy arrays in/out. 5 tests incl. CPU==GPU cross-validation. - ThreadFusion (thread_fusion=True): emits TransformDPP<GPU_NVIDIA, TF::ENABLED>. Auto-falls back to scalar for shapes whose row bytes are not 16-aligned: external tight-pitch pointers fault on vectorized loads (found via real 'misaligned address' on a 251-wide image). 5 tests: exactness vs scalar baseline, odd widths, uchar3, distinct cache keys. - Multi-GPU: DeviceBuffer/CircularTensor take device=N (per-device primary contexts retained via the driver API); compose pipelines follow the input's device (torch cuda:N / cupy / DeviceBuffer); DLPack device id propagated. Honest caveat: single-GPU box, exercised on device 0 with clean errors for absent devices. - Torch integration suite (torch 2.12+cu130, real GPU): zero-copy both directions (mutation-visibility proven), external torch streams, out=, uchar3 HWC->CHW vs torch reference, CircularTensor->torch window, batch HF from torch list. Found+fixed _alloc_out shape bug for TensorSplit chains (was returning HWC-shaped tensor for planar CHW output). - fk::Equal / Tuple-input ops: documented as structural limit (needs a multi-source Read producing Tuple<A,B>; upstream feature candidate). - spikes/repro_divergent_sum_z.cu: reproducer behind upstream issue #250 (Divergent Executor launches sum-of-z planes; compute-sanitizer verified OOB + canary corruption). Full matrix: 13 suites x 2 backends (nvcc+clang) + cpu + torch, all green.
morousg
approved these changes
Jun 11, 2026
morousg
pushed a commit
that referenced
this pull request
Jun 24, 2026
…yone DivergentHF: fix off-by-one in sequence selector (0-based kernel)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #1 (thanks for the fast merge!) with the work that landed after the merge cut:
1. Real end-to-end example: YOLOv8 inference (
examples/11_yolo_inference.py)JPEG -> ONE fused GPU kernel (letterbox
Resize(640,640, PRESERVE_AR, bg=114)+Div(255)+TensorSplitHWC->CHW) -> YOLOv8n ONNX -> NMS -> boxes. Verified onbus.jpg: 4 persons + 1 bus, correct boxes; fused preprocess ~0.55 ms steady-state.examples/fetch_yolo_assets.shdownloads the assets.2. CircularTensor hardening (one real bug fixed)
Bug: generated
ct_update/ct_snapshotbaked W/H as C literals while the cache signature doesn't include size -> two same-signature CTs with different sizes cache-hit the wrong .so (OOB, verified). Fix: sizes live in the PyCT struct, read at runtime (the values-vs-types rule applied consistently). Plus: race audit (seq1's written temp slot proven disjoint from seq2's read slots for both orders × all rotation indices), temp/data coherence across 3 wraps,generate_source()API, and CI now compile-checks CircularTensor codegen too. 21 checks.3. CPU backend (
target="cpu")Same fused chains through
TransformDPP<ParArch::CPU>on host memory, numpy in/out, no CUDA required. Compiled with clang++ (note: g++ rejectsStream_<ParArch::CPU>'s ctor spelling — possibly worth an upstream look). CPU==GPU cross-validated.4. ThreadFusion (
thread_fusion=True)Emits
TransformDPP<GPU_NVIDIA, TF::ENABLED>. Auto-fallback to scalar when row bytes aren't 16-aligned (external tight-pitch pointers fault on vectorized loads — found the hard way with a prime-width image).5. Multi-GPU device routing
device=Non DeviceBuffer/CircularTensor; pipelines follow the input's device; DLPack device id propagated. Honest caveat: developed on a 1-GPU box, exercised on device 0.6. Torch integration suite (real torch 2.12+cu130)
Zero-copy both directions (mutation-visibility proven), external torch streams,
out=, HWC->CHW vs torch reference, CircularTensor->torch, batch HF from a torch list. Found+fixed an_alloc_outshape bug for TensorSplit chains.Also includes
spikes/repro_divergent_sum_z.cu— the reproducer behind FusedKernelLibrary#250.Verification
13 suites × 2 compilers (nvcc + clang) + cpu suite + torch suite, all green on RTX PRO 6000 (sm_120, CUDA 13.3) against LTS-C++17.